doc: add an initial compute autoscaling RFC - #197

@savme what's it look like to leverage the HPA controller? We would just need to run a small prometheus deployment in the cluster for instance metrics?

@scotwells assuming we'd run stock HPA at the edge, I think Prometheus would only be one piece. We'd also need to:

• implement the WorkloadDeployment scale subresource (including a selector that maps each deployment to its backing pods)
• make sure Karmada doesn't fight autoscaling decisions by adding custom retention for HPA-owned fields on WorkloadDeployment
• run a metrics adapter HPA can query for kraftlet-managed Pods, likely Prometheus + prometheus-adapter via custom metrics
• create an HPA per WorkloadDeployment scaling policy

@savme nice! That seems like a pretty clean path.

Thoughts on trying that before we reach to re-implementing the capabilities HPA gives us?

@scotwells good call! The RFC is close enough to HPA, so this is definitely worth prototyping first. I think most of the timing defaults that may not work well for us are probably tunable anyway. I'll try an HPA-based approach so we can see what the integration looks like and whether we're happy with the scaling latency/behavior. I'll update the doc based on what we find

Linking related work here: datum-cloud/unikraft-provider#54

Curious if you have any HPA findings to report @savme

Hey @0xmc 👋

Definitely! The biggest thing to report is: kraftlet does not currently provide a straightforward way to ingest metrics that would be compatible with metrics.k8s.io (which is a requirement for a resource-based HPA), so we'll need a workaround to expose metrics it does provide in a compatible format.

I'd love to run the full ingest pipeline by you when you have a few minutes. My current sense is that the best path forward is a small exporter that exposes metrics from ukpd (https://unikraft.com/docs/integrations/sdks/go#instances) (mapping ukpd instances to pods) alongside a prometheus-adapter registered as a metrics-server for the cluster.

I'm basing this on Scot's work in datum-cloud/unikraft-provider#54. You can check out the implementation in the linked PRs on datum-cloud/enhancements#799.